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Introduction 


Facing with the fast development of science and technology, it has been 
widely recognized that students should engage in real-world tasks from an 
interdisciplinary curriculum, combining theory and practice, and develop 
their skills in creative problem solving. In response to the needs, Science, 
Technology, Engineering and Mathematics (STEM) curriculum model was 
proposed and one of the most significant and distinguishing curriculum 
characteristics of STEM compared to other discipline domains is hands-on 
skill development. 

Hands-on skill (practical skill) development can bridge the gap between 
theory and practice as well as enhance problem solving skills. Dufresne, 
Gerace and Leonard (1997) have proposed a model for the problem solving 
process, identifying three key knowledge essentials of the process: conceptual 
knowledge (CK), operational/procedural knowledge (PK), and problem-state 
knowledge (PSK). Hands-on skill (practical skill), which includes concept 
knowledge (CK) and procedural knowledge (PK), has been acknowledged as 
the core competence in technology education and is defined as the capability 
to utilize the skills, techniques, and engineering tools for technology practice 
(Barlex, 2007; McLaren, 2007). Hands-on science could promote learning 
transference because it echoes with the concrete-to-abstract nature of cog- 
nitive development, providing additional sources of brain activation via kin- 
aesthetic involvement and increasing students’ motivation and engagement 
(Flick, 1993; Klahr, Triona & Williams, 2007). Clough and Kauffman (1999) had 
encouraged students to make repetitive “connections” between concepts in 
various of contexts and applications, to enhance their problem-solving skills. 
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Flick (1993) pointed out that hands-on activities usually emphasize students’ capacities of logical, mathematical, 
linguistic, and spatial intelligences. Unal (2008) revealed that hands-on activities did have a significant positive 
effect to help students to replace their misconceptions with correct ones. Furthermore, the hands-on activity, 
trial-and-error experiential learning helped students enhance their creative skills and problem-solving abilities as 
well as realize the value of collaboration (Shieh & Chang, 2014). Undergraduate students’ participation in hands-on 
activities is widely believed to encourage them to pursuit of advanced degrees and careers in STEM fields (Russell, 
Hancock & McCullough, 2007). Moreover, internship and mentoring programs in STEM fields can foster student 
interest and provide valuable hands-on experience (Christie et al., 2008). 

It is true that students need to acquire CK and PK that are utilized during their design and/or problem 
solving tasks. In addition, CK and PK have to be taught in an active manner and be modelled by teachers, experts 
or fellows (Sidawi, 2009). However, the connection between theory and practice for technology education needs 
to be established firstly because CK brings up the use of PK (McCormick, 1997). In exploring how novices and 
experts solved mechanical problems, Johnson (1997) found out that “The better performance of the experts was 
attributed to their deeper conceptual understanding of physics principles” (p. 164). Therefore, if education ignores 
students’ comprehension of CK, it is highly probable that students cannot solve problems independently and the 
Einstellung Effect (Set Effect) might occur (Luchins, 1942). Failures in delivering CK result in the inflexibility of the 
students to deal with novel problems, which is harmful to skill transference (Gagne, Yekovich & Yekovich, 1993). 
Similar results were reported in the studies upon troubleshooting in the electrical and mechanical systems (Case 
& McKeough, 1990; Means & Gott, 1988). Many studies have indicated that students’ conceptual understanding 
is Significantly associated with their problem-solving ability (Dym, Agogino, Eris, Frey & Leifer, 2005; Gagne et al., 
1993; Luchins, 1942; Lutgens & Mulder, 2002). As a result, the level of CK understanding as well as the interaction 
of CK and PK enables to effect the successful transfer of knowledge. 

In response to the STEM trend, the Ministry of Education (2003) in Taiwan announced a nine-year curriculum 
guideline for 1-9 grade schools in which “nature and living technology” and “mathematics” emerged as the imple- 
mentation of STEM course, to cultivate the students’ science and technology literacy. The content of the nature 
and living technology curriculum includes knowledge and abilities related to materials, energy, life science, the 
planet, environmental science, wildlife and nature preservation, the use of information technology, science and 
scientific studies, and respect for life. The mathematics curriculum includes the basic concepts of numbers, patterns, 
and quantities; calculation ability; reasoning and critical thinking ability; and the ability to discuss mathematics 
with others (Ministry of Education, 2003). However, since them, STEM teaching and learning which emphasized 
students’ position as the center of learning activities (Lou, Shih, Diez & Tseng, 2011), were still in their cradle stage 
in Taiwan (Tseng, Chang, Lou & Chen, 2013). Two reasons will be addressed as follows. 

(1) Lack of the chance to put theory into practice 

In Taiwan, some studies emerged on integrating STEM into experiments in teaching (Chang, 2009; Chang, 
Chen, Kuo & Shen, 2011; Chang, Wu, Kuo & You, 2012; Tseng et al., 2013; Chang, Yu, Kuo, Mai & Chen, 2015) have 
found that most students show positive attitudes toward STEM course, but they had no chance to understand the 
interrelationship of STEM or to put theory into practice, though they acquired more technology-related knowledge 
at school. Moreover, mathematics was found to be the least popular subject within STEM. 

The vocational education of hands-on skill is creating educational structures of CK and PK to deliver 50-minute 
lectures, and helping students acquire skill in practicum. Students’ achievements of hands-on skills are limited 
because the teaching paradigm and students’ individual difference, though remedial learning was developed to 
solve these problems. However, remedial education often carries a stigma because of its association with under- 
prepared students. 

(2) Overemphasize PK teaching and operation training 

The mission of Technological and Vocational Education (TVE) in Taiwan seeks to cultivate technical manpower 
for the country; hence, vocational education programs emphasize students’ hands-on skills as well as problem 
solving abilities (Ministry of Education, 2006). Senior vocational high school in Taiwan acts in concert with the 
economical policy of government-providing skilled labor for the small and medium enterprises in 1960s, offering 
the quantity and quality of labor for the capital and technology-intensive industries in 1970s, and supplying higher 
levels of technological and business personnel for internationalization and the open market in 1980s (Ministry 
of Education, 2015). Therefore, senior vocational high school in Taiwan is determined to equip students with 
technical skills to complete the tasks of a specific job and offers courses in diverse areas such as industry, com- 
merce, agriculture, marine products, home economics, opera and arts etc. Nowadays, students are encouraged to 
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obtain professional certificates and credits to enhance their competitiveness by participating in skill competitions 
(contests) for entering a higher education. Consequently, some technology teachers seem to overemphasize PK 
teaching and operation training, ignoring students’ comprehension of CK and the connection between cognition 
and metacognition, which might become the obstructions for students’ hands-on skill acquisition (Dym et al., 
2005; Lutgens & Mulder, 2002). 

Therefore, how to generate effective hands-on skill learning system for STEM courses for senior vocational 
high students by enhancing the interaction of CK and PK to clarify the misconception of students is an important 
issue. 

Due to the robust development of Information Communication and Technology (ICT), the development of 
computer-assisted learning systems (CALS), which adapts users’ features such as knowledge structure, background, 
preference and interest to meet the individual needs of the students, becomes promising and feasible (Lazarinis, 
Green & Pearson, 2010; Lin, Liu & Yuan, 2001; Liu, Lin, Chiu & Yuan, 2001). Meanwhile, CALS not only alleviates the 
burdens of STEM faculties, but also offers sufficient resources for students. Additionally, it is important to develop 
a CALS to diagnose student's individual learning difficulties and misconception; the benefits of which also include 
providing educational supports and guidance/remedial learning services based on the diagnosis (Chen, 2011; 
Hsiao et al., in press; Hwang, Panjaburee, Triampo & Shih, 2013). The comparison of expert and student knowledge 
structures could serve as a key reference for efficient process to diagnose student's individual learning difficulties 
and misconception (Appleby, Samules & Treasure-Jones, 1997; Brown & Burton, 1978; Chang, Liu & Chen, 1998; 
Shih, Kuo & Liu, 2012; Wenger, 1987; VanLehn, 1988; Wu, Kuo & Yang, 2012). 

Although people generally consider constructing a computer-assisted learning environment to be expensive, 
complicated and time-consuming, more evidences are still to convince educators of educational efficiency of 
CALS. With the attempt to shorten the gap between theory and practice and tackle the problem regarding lack of 
an educational Paradigm in STEM field. Therefore, this research proposed and implemented an innovative STEM 
course with CARLS support to help students clarify their misconceptions and enhance the interaction of CK and 
PK in the STEM course. By applying the Knowledge Structure-based Adaptive Testing algorithm (Shih et al., 2012; 
Wu et al., 2012), this proposed CARLS has two distinguishing features: (1) CARLS, significantly reducing the number 
of test items compared with traditional paper and pencil test, can provide different test items according to indi- 
vidual differences in prior knowledge of students. (2) Test items of CARLS focusing on the interaction of CK and PK 
different from those of traditional paper and pencil test enhance students’ hands-on ability. From an educational 
viewpoint, students do not really acquire mastery of hands-on skill until they are able to utilize the interaction 
of CK and PK to solve problems successfully. Accordingly, the proposed STEM course with CARLS support would 
enhance the interaction of CK and PK to strengthen students’ hands-on ability. The current research addressed 
the following issues: 

1. The effectiveness of the proposed STEM course with CARLS support on the knowledge achievement; 

2. The effectiveness of the proposed STEM course with CARLS support in enhancing hands-on skill per- 
formance; 

3. The students’ technology acceptance attitude toward CARLS. 


Computer-assisted Remedial Learning System 


Traditional Science and Technology Education (STE) is concentrating on gaining appropriate professional 
knowledge and skills. Pedagogical process usually is based on memorization of important selected knowledge and 
learning formal algorithms for corresponding use of them in practice (Broks, 2011; 2014). Developing an effective 
remedial learning system to inspire scientific thinking as a backbone of new STE is an issue. 

With the vigorous development of computer and internet technology, development of teaching/learning 
system refocuses on adapting users’ features such as goals/tasks, knowledge, background, hyperspace experi- 
ence, preference, and interests to meet the individual needs of students (Lazarinis et al., 2010; Lin et al., 2001; 
Liu et al., 2001). It is important to develop a CALS wherein testing items and procedures could be personalized 
according to student’s individual performance, prior knowledge, goals and preferences (Chen, 2011; Hsiao et al., 
in press; Hwang et al., 2013). Moreover, the key issue is how to diagnose student’s individual learning difficulties 
and misconceptions as well as to provide educational supports and guidance/remedial education based on the 
above diagnosis. The comparison of expert and student knowledge structures is an efficient process to diagnose 
students’ individual learning difficulties and misconceptions (Appleby et al., 1997; Brown & Burton, 1978; Chang, Liu 
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& Chen, 1998; VanLehn, 1988; Wenger, 1987; Shih et al., 2012; Wu et al., 2012). Therefore, a computerized adaptive 
testing system (Shih et al., 2012; Wu et al., 2012) for this proposed STEM course was constructed to help students 
detect misconception, remedy it, and, hopefully, enhance their hands-on skills. 

In traditional paper and pencil test item for Conceptual Knowledge, students are asked questions, e.g. If the 
block (open/short) is short between the node E and F in this circuit (Figure 1), what is current |,, to show their 
understanding of knowledge. However, CARLS provides students with test item in form of the interaction of CK 
and PK. With the same question, CARLS would not only ask students to calculate the |,, but also measure it with 
multimeter operation. Therefore, the question will be revised to be: How do students apply multimeter to measure 
current I, (Figure 1)? Firstly, students are able to apply the principle to calculate |,, and decide the block (open or 
short) is open or short between the node E and F. Finally, students use the multimeter to measure I,, and ensure 
the calculation value. 
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Figure 1: Calculation of the current. 


In the research, 20 industrial electronics experts were invited to construct the expert knowledge structure 
as well as to modify the item bank of CARLS from Industrial electronics C Technician Skills Certification Test (Skill 
Evaluation Center, 2013). Figure 2 illustrates the expert knowledge structure of “electronics meter operation and 
measurement skill”. The lower nodes represent the fundamental concepts; the upper nodes, which are more com- 
plicated concepts than lower ones, are the integrated/interaction concepts of lower ones. 
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Instrument operation and measurement 


1 Measurement of 2 Measurement of 3 Measurement of 
multimeter (ACV) multimeter (DCV) multimeter (DCmA) 
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Figure 2: Expert knowledge structure of industrial electronics meter operation and measurement. 


The student knowledge structure was constructed according to the 215 students’ responses to the item bank 
via CARLS in accordance with the expert knowledge structure. The relation of the sequence in each level of the 
students’ knowledge structure was computed by the item structure algorithm. For example, the students’ structured 
knowledge was built up by the algorithm to diagnose their misconceptions and personalized the remedy paths. The 
reliability and validity of the item bank were analysed; meanwhile, the parameters of Item Response Theory (IRT) 
such as difficulty, discrimination, and estimation, were calculated for the reference of item selection and revision 
(Chang, 2009). The Cronbach a for the whole test was 0.772, which met the minimal requirement criteria (over 0.7) 
for internal consistency of test (DeVellis, 1991; Nunnally, 1978). 

According to the expert knowledge structure, to achieve the learning objective of “1 Measurement of multi- 
meter (ACV)", students have to complete the learning unit in the following sequence, starting with “1-1” and then 
“1-2” Similarly, to accomplish the learning objective of”1-1”, students need to complete the sub-skill learning units 
in the following sequence, “1-1-1” and then “1-1-2” Therefore, the sequence of test item in traditional assessment 
from the upper to lower concepts is, 11-1 1-2-4 1-1-1-91-1-2-91-2-1-9 1-2-2 1-2-3. 

CARLS adopted the student knowledge structure to diagnose student's misconception and personalized the 
remedial learning. If a student answers the upper item precisely (without misconception), he/she does not have to 
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answer, the lower items affiliating with the upper one; if he/she makes an error on the upper item, he/she should have 
some misconceptions for the lower items affiliating with the upper one. Accordingly, the items and the sequence 
of item testing provided by CARLS were created. Figure 3 showed students A's sequence of test items and remedial 
learning path calculated by CARLS according to the A‘s responses. Student A had the misconception of“2-3-1", which 
led to the upper node error. Therefore, student A’s remedial learning path should be 2-3-1—>2-3—>2. 
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Figure 3: Student A’s remedial learning path. 


The design concept of the proposed CARLS is listed as follows. 

1) By challenging the students, starting with single-concept problems and gradually progressing to 
multi-concept problems, and by making repetitive connections between the different concepts, 
students can be trained to apply concepts learned in different places and times to solve problems in 
new contexts. 

2) Compared with traditional paper and pencil test, CARLS were designed to reduce the number of 
test items and provide different test items according to individual differences in prior knowledge of 
students. 

3) Test items of CARLS were designed to focus on the interaction of CK and PK to enhance students’ 
hands-on ability. 
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Methodology of Research 
Participants and the Experimental Setting 


To evaluate the effectiveness of the proposed STEM course with CARLS support, a pre-experimental design 
of one-group pre-test/post-test was employed in this research due to the limitation of students’ enrollment in 
the proposed course. A total of 32 students (11" grade) from the Department of Electronics of a vocational high 
school in central Taiwan voluntarily participated in this research. All students of the class completed all the steps 
and procedures in the school timetable, and the sessions were implemented in 10 weeks. 

According to the one-group pre-test/post-test design (Campbell & Stanley, 1963), no control group or baseline 
was compared with and thus the findings of the research should be carefully interpreted. However, Yang (2007) ad- 
opted the pre-experimental design of one-group pre-test/ post-test to reveal the effect of promoting students critical 
thinking skills; Sak and Oz (2010) also adopted this design to investigate the students’ creative thinking activities. 


Procedure and Treatment 


Table 1 showed the experimental procedure. In the first two weeks, the teacher explained the objectives and 
went through the syllabus of the proposed course, as well as the fundamental of the electronics (CK) including 
the symbols of electrical component and basic equations. In the next two weeks, students were taught how to 
operate multimeters for measuring voltages and currents in electrical devices (PK). Figure 4 showed that students 
had to master the operation of the multimeter for different purpose such as measuring DC or AC voltage, DC cur- 
rent, resistance, diode, etc. In week 5-6, the teacher taught the interaction of CK and PK for students how to design 
the circuit layout with the right electronic units and calculate the right data. In the first 6 weeks, students were 
required to finish the instruction material (films) in advance before the class and teacher help students to solve 
their difficulties in the class. 

After the six-week sessions, to explore the students’ prior knowledge and skills, a pre-test exam was administered. 
Then, after introducing the functions of CARLS to students, the teacher acted as a facilitator, to help students to 
proceed with the remedial learning through CARLS. Finally, a post-test was conducted in the 10" week. 


Table 1. |§ The experimental procedure and school time table. 


Procedure Description School time table (week) 
Instruction 1 The teacher taught the CK 1-2 

Instruction 2 The teacher taught the PK 3-4 

Instruction 3 The teacher taught the interaction of CK and PK 5-6 

Pre-test Students took a CK and PK test 

Remedial learning Students proceeded the remedial learning through CARLS 7-10 

Post-test Students took a CK and PK test through CARLS 





Figure 4: Operating a multimeter to measure the Voltage and Current in electrical appliances. 
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Data Analysis 


As shown in table 2, the research measured students’ knowledge, achievement, hands-on skill achievement 
and attitude by three evaluation tools, the tests for the interaction of CK and PK, the operational test of hands-on 
skill, and the technology acceptance attitude of students toward CARLS, respectively. The collected data were 
analysed by descriptive statistics, t-test, and ANOVA. 

The pre-test/post-test for the interaction of CK and PK were selected from the Industrial electronics C Techni- 
cian Skills Certification Test (Skill Evaluation Center, 2013) and the operational test of hands-on skill was selected 
from the “C test of the Industrial Electronics Meter Operation and Measurement Skill” (Skill Evaluation Center, 2013). 
The tests for the interaction of CK and PK as well as the operational test for hands-on skill were modified according 
to a strict procedure by 20 industrial electronics experts, so the expert validities are reasonably fair. 


Table 2. Description of research measurement. 


Objective of evaluation Evaluation instruments Description of the instruments 
Knowledge Industrial electronics C Technician Skills Certifica- _ Examine the students’ interaction of CK and PK 
tion Test 
Skills C test of the Industrial Electronics Meter Operation Evaluate the operation and measurement skills by 
and Measurement Skill teacher rating scores 
Attitude Technology Acceptance Model (TAM) toward Examine students’ attitudes toward CARLS 


CARLS questionnaire and in-depth interview 
Results of Research 


Three parts in research results are to be discussed concerning the evaluation of the proposed STEM course 
with CARLS support: (1) the effectiveness of the proposed course with CARLS support upon the enhancement of the 
knowledge, achievement; (2) the effectiveness of the proposed course with CARLS support upon the enhancement 
of the hands-on skill performance; and (3-1) the technology acceptance attitude of students toward CARLS; (3-2) the 
differences of technology acceptance attitude toward CARLS between the high score and the low score groups. 


The Effectiveness of STEM Course with CARLS Support Upon the Enhancement of the Knowledge Achievement 


Mean scores of the pre and post test as well as the result of t-test is shown in table 3. Mean scores of the pre- 
test and post-test were 76.56, and 84.75, respectively. Result from t-test as revealed in table 3, demonstrated that 
effectiveness of the proposed course with CARLS support upon the enhancement of the knowledge achievement 
is Significant (t=7.96, p<.000). Furthermore, this research adopted, the effect size d to measure the strength of the 
proposed course with CARLS support upon the enhancement of the knowledge achievement (Cohen, 1988). In 
Cohen's definition, 0.2<d<0.3 means a small effect size, around 0.5 means a medium effect size, and d>0.8 means 
a large effect size. In table 3, the Cohen's d value of 1 indicates a large effect size, suggesting a great help from the 
proposed course with CARLS support. 


Table 3. t-test of pre-test/post-test of students. 


N Mean SD df t value p Cohen’s d 
Pre-test ed 76.56 8.84 
ol 7.96 .0001*** 1 
Post-test OZ 84.75 7.18 


The Effectiveness of STEM Course with CARLS Support upon the Enhancement of the Hands-On Skill Performance 
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A one-way ANOVA was conducted to compare the mean difference of the hands-on skill performance among 
three different groups, high, medium and low scores, which were assigned according to the scores of post-test. 
Table 4 illustrated that difference of means among high, medium and low scores groups was significant (F=4. 963, 
p<0.05). In addition, effect size (n* = 0.255) was also examined and showed approximately large (n* > 0.26) effect 
size (Cohen, 1992). Nevertheless, pairwise comparison (Scheffé) revealed that there was no significant effectiveness 
between high and medium score groups (p=.994). The reason could be that students in high and medium score 
groups might already have achieved well interaction of CK and PK, so they have better hands-on skill performance 
after the proposed course. These results partially supported that the proposed course with CARLS support could 
assist in the enhancement of students’ hands-on skill performance. 


Table 4. |= ANOVA for hands-on skill performance of the high, medium and low score groups. 


Groups N Mean SD F p Post-Hoc (Scheffe) n2 
High score 9 96 0 
Medium score 14 94.86 2.905 4.963 014* 1>3; 2>3 0.255 
Low score 9 64 48 


Technology Acceptance Attitude of Students toward CARLS 


The mean score of Perceived Usefulness of CARLS on the Technology Acceptance Survey was 3.80, which is 
greater than the median of 3. Table 5 revealed that students regarded CARLS as a beneficial channel to enhance 
their learning. In particular, high mean scores of some items such as Q1 (mean=4.06), Q4 (mean=4.09), and Q7 
(mean=4.06) demonstrated that students considered CARLS an efficient method to examine (or evaluate) their levels 
of understanding and a helpful skill acquisition system. On the other hand, items with low mean scores such as 
Q3 (mean=3.25) and Q5 (mean=3.41) implied lower effectiveness of CARLS in enhancing their learning motivation 
and presenting the detailed explanations of their misconceptions. 


Table 5. Perceived usefulness of CARLS technology acceptance. 


Dimension Item Content of the Questionnaire Item mean (SD) 
Q1 CARLS is helpful for my learning. 4.06 (.504) 
Q2 CARLS enhanced my learning effectiveness. 3.81 (.592) 
Q3 CARLS facilitated my learning motivation. 3.20 (.003) 
Perceived Useful- Q4 CARLS detected my comprehension status of the interactions of CK and PK. 4.09 (.641) 
NESS Q5 CARLS provided detailed explanations of my misconceptions. 3.41 (1.073) 
Q6 CARLS corrected my misconceptions. 3.91 (.893) 
Q7 CARLS is positive for overall learning. 4.06 (.619) 
Overall 3.80 (0.81) 


As to Perceived Ease-of-Use of CARLS in the Technology Acceptance Survey, the mean of the total items was 
3.69, greater than the median of 3. Table 7 showed that students regarded CARLS as a system easy to apply based on 
the response in Q1 (mean=4.06) and Q7 (mean=4.06). Contrarily, the lowest mean score of Q2 (mean=3.16) implied 
that students expressed less agreement upon the convenience of CARLS while compared with its paper-and-pencil 
counterpart. As for Q6 (mean=3.56), leaner’s responses demonstrated less agreement on CARLS as an easy task. 
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Table 7. Perceived ease-of-use of CARLS technology acceptance. 


Dimension Item Content of the Questionnaire Item mean (SD) 

Q1 It is very easy for me to operate CARLS 4.06 (.878) 

Q2 It is more convenient to use CARLS than using the traditional paper and pencil test. 3.16 (1.110) 

Q3 CARLS is a very easy testing system to use. 3.78 (.870) 

Perceived Ease- Q4 CARLS is quite clear and comprehensible to use. 3.63 (.751) 
of-Use Q5 It is easy to use CARLS for remedial learning. 3.56 (.669) 

Q6 It is very convenient to use CARLS for testing and remedial learning. 3.06 (.716) 

Q7 It is efficient to use CARLS. 4.06 (.759) 

Overall 3.69 (.87) 


The Difference in Students’ Technology Acceptance Survey of CARLS between High and Low Score Groups 
Attest was utilized to examine the differences in Technology Acceptance Survey of CARLS between high and 
low score groups. The result showed no significant difference in both Perceived Usefulness and Perceived Ease-of- 


Use of CARLS as illustrated in Table 8. 


Table 8. t-test of perceived usefulness and ease-of-use of CARLS between two groups. 


Item Group N M S.D. t value p 
High score 9 3.94 A17 
Perceived Usefulness 846 410 
Low score 9 3.76 A457 
High score 9 3.79 429 
Perceived Ease-of-Use 852 407 
Low score 9 3.59 587 


Discussion 


This research employed a STEM course with CARLS support for vocational high school students in Taiwan to 
enhance their knowledge achievement, and hands-on skill performance. After statistical analyses, including single 
group ttest and ANOVA, results indicated that CARLS is effective and contributed in enhancing students’ knowledge 
achievement and hands-on skill performance. Moreover, while examining participants’ attitude toward CARLS by 
using the Technology Acceptance Survey, the current research found out that over 50% of the students considered 
CARLS easy to operate and useful in clarifying their misconceptions. Students who learned through the proposed 
STEM course with CARLS support generally had a positive learning experience, perceived their use of CARLS as 
satisfying, useful and easy and also expressed a continuing intention of using CARLS. Some advantages conveyed 
by the participants through interviewing the teacher and students after using CARLS are listed as below. 

(1) CARLS could benefit educator in the following ways. 
(a) CARLS helped the educator comprehend the individual differences of students through the learning 
portfolios. 
(b) CARLS assisted in evaluating whether students have achieved the mastery level of the interaction of 
CK and PK as well as locating what their learning difficulties are in skill performance through students’ 
learning portfolios. 
(c) CARLS had the merit of time-saving in educator test preparation so that he/she may pay more attention 
to individual learning progress as well as to provide extra training materials for unprepared students. 
(2) CARLS could benefit students in the following ways. 
(a) Ithelped students diagnose their misconceptions that allowed individual learning journey to take place 
based on individual difference while using CARLS independently. 
(b) It effectively enhanced students’ hands-on practice according to their individual leaning status and 
needs. For students with high level of mastery in the interaction of CK and PK, CARLS saved their time 
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in implementing hands-on practice; whereas it guided low mastery level students to do their individual 
remedial education without pressure. 
(c) It promoted students’ self-efficacy along their cognitive stage to consolidate skill acquisition. 

In terms of the Technology Acceptance of CARLS, over 50% of the students previewed CARLS easy to operate 
and useful in clarifying their misconceptions. Nevertheless, only a small part of the participants (25%) indicated 
that CARLS escalated their motivation to utilize it. Hence, some improvements on CARLS are suggested based on 
rest (75%) of the students. 

(1) Facilitating student motivation 
Researchers pointed out the learning effectiveness may be affected by student motivation (Hamjah, Ismail, 
Rasit & Rozali, 2011; Lim, 2004). Since the design of CARLS took its root in the diagnostic and remedial 
learning system, which is often labelled as“the system designed for the low academic achievement learner’, 
it probably eliminated students’ motivation. Therefore, futures studies might focus on developing CARLS 
that facilitate the students’ motivation to encourage a willingness to learn. 

(2) Providing adaptive remedial materials for low achieving students 
Students’ interview indicated that CARLS failed to effectively provide satisfying explanations of misconcep- 
tions, especially for low achieving students. Those students frequently experienced learning frustrations, 
which resulted in their low motivation. However, besides remedial materials, they probably needed more 
assistance and guidance to adjust their learning strategies to self-regulation. Therefore, CARLS should be 
modified to provide appropriate teaching strategies and more adaptive materials for them. 

Regarding to evaluation of Perceived Usefulness and Perceived Ease-of-Use of CARLS, the results indicated no 
significant difference between the high and low score groups. This research inspiringly revealed that students per- 
formed CARLS equally well on the knowledge, achievement to apply hands-on skill on problem-solving regardless 
of what group they were in. The results mirrored with findings from previous research comparing the effectiveness 
of online and paper-and-pencil tests in learning performance (Anakwe, 2008; Bugbee Jr, 1996; Clariana & Wallace, 
2002; Zandvliet & Farragher, 1997). However, slightly over one third of the students (37%) remained their custom 
in using paper and pencil tests. 


Conclusions 


This research proposes a STEM course with CARLS support for vocational high school students in Taiwan to 
enhance their knowledge achievement, and hands-on skill performance. The results of the research revealed that: 
(1) students with better understanding of how CK and PK interact will benefit in clarifying their misconceptions, 
which probably improve their knowledge achievement on CK and/or PK further; (2) only after successfully devel- 
oping an understanding of, CK, PK, and how CK and PK interact was the students able to acquire hands-on skill in 
the proposed STEM course with CARLS support; (3) The reasons causing learning difficulties of low achievement 
students are complicated and require teacher's counseling and help individually. This research has provided pre- 
liminary empirical evidence to support the effectiveness of a STEM course with CARLS support in terms of facilitat- 
ing students’ understanding of interaction of CK and PK to acquire hands-on skill for problem-solving in the real 
world. Furthermore, the proposed STEM course with CARLS support does not only help the educator comprehend 
the individual differences and misconceptions of the students through their learning portfolios, but also saves 
the educator's time for paying more attention to the individual's learning difficulties as well as to providing extra 
training materials for unprepared students. 

Accordingly, some suggestions are made based on the findings of the current research: 

(1) The high score group used CARLS more frequently than its low score counterpart; whereas additional 
cognitive labour might be present for the low score group students since they had to concentrate not 
only on knowledge acquisition but also on mastery themselves of the operation of CARLS. To avoid 
cognitive overload of the low score group, this research recommends that low score students better 
focus on CK acquisition first, and then proceed to CARLS once they managed to learn CK. 

(2) More scaffolding examples for conception clarification, e.g. Dual Situated Learning Model (DSLM) (She, 
2002; 2003; 2004a, b) should be added to CARLS for knowledge consolidation instead of knowledge 
over-generalization. Furthermore, the practicing example, in CARLS should be embedded and scheduled 
into course design in the first place to help students achieve hands-on skill training. 

(3) Itmight bea debate as to whether higher level cognitive process really occurs as students learn through 
CARLS. 
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Some limitations of this research should be presented. In this research, a pre-experimental design of one-group 
pre-test/post-test was conducted, without baseline or comparison with the control group, and thus the research 
findings should be carefully interpreted (Campbell & Stanley, 1963). Future studies might address the following 
issues. First, the findings and implications are obtained from merely one particular learning system; moreover, 
representativeness of the studied sample is limited since the sample size was not big. Second, individual differ- 
ence variables such as personality and learning style, could be incorporated in future studies. Third, the students in 
Taiwan have unique value and behavioural patterns, such as they had diverse definition of learning achievement 
and remedial learning. 
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